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$ SUMMARY 


\ 

The  goal  of  the  over-all  research  program  has  been  the  measure- 
ment of  operator  performance  in  the  ASW  weapon  system.  The  ability 
to  make  these  measurements  is  of  practical  importance  as  well  as  of 
theoretical  interest.  Three  previous  partial  reports  have  been  pub- 
lished covering  the  work  in  the  program.  Those  reports  describe  the 
development  and  validation  of  a set  of  ASW  activity  scales  for  the  eval- 
uation of  operator  activities  in  the  ASW  attack  team. 

The  question  arose  as  to  the  possibility  of  using  these  scales  as  an 
aid  in  coaching  of  team  members  during  ASW  attack  exercises.  It  was 
felt  that  coaching  in  this  manner  would  bring  about  an  improvement  in 
ASW  team  performance. 

It  was  noted  early  in  the  research  program  that  coaching  played 
an  important  part  in  the  performance  of  the  ASW  team.  It  is  an  impor- 
tant activity  of  several  officers  in  the  team.  A theoretical  discussion 
of  coaching  in  the  Second  Partial  Report  pointed  out  a series  of  speci- 
fications for  couching  of  operators.  The  use  of  the  activity  scales  by 
enlisted  coaches  furnishes  a methodology  for  satisfying  these  specifi- 
cations. 

Procedure 


Regular  dual-ship  attack  teacher  exercises  were  conducted  for 
ships’  teams.  The  ships  were  divided  into  two  groups,  one  group  being 
coached  in  their  activities,  the  other  carrying  out  standard  procedures 
without  coaching.  Experimental  conditions  were  carefully  controlled 
so  as  to  maintain  similar  conditions  for  the  groups.  Ratings  were  made 
of  operator  activities  for  both  groups  and  the  attack  error  was  measured 
to  serve  in  the  evaluation  of  team  performance. 
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He  suits 

It  was  found  that  the  'wo  groups  of  ships'  teams  were  approximately 
•qual  in  attack  performance  during  the  equating  trials.  Aftc’-  exposure 
to  the  experimental  conuitions,  the  coached  group  showed  significant  im- 
provement as  compared  to  the  control  group.  This  improvement  was 
indicated  in  several  ways: 

1.  The  coached  group  improved  11  yards  in  mean  attack 
error,  whereas  the  control  group  decreased  in  per- 
formance by  11  yards. 

2.  The  coached  group  made  a significantly  higher  propor- 
tion of  lethal  attacks  than  the  control  group.  Although 
the  coached  group  was  9%  superior  to  the  control  group 
during  the  equating  trials,  it  showed  a 17%  advantage 
during  the  experimental  sessions. 

3.  The  ratings  of  individual  operators  showed  a consistent 
improvement  for  the  coached  teams  whereas  the  con- 
trol group  ratings  seemed  to  fluctuate  randomly. 

The  results  of  the  study  were  found  to  be  statistically  significant  at 
a fairly  high  level  of  confidence.  However,  a repetition  of  the  experi- 
ment is  warranted  before  broader  generalizations  may  be  made  from 
the  findings.  Such  a study  is  included  in  the  planning  for  future  inves- 
tigations. 

Future  Research  Plans 


A repetition  of  the  present  study  is  planned  for  the  purpose  of  con- 
firming the  obtained  results.  This  study  will  once  again  be  confined  to 
the  evaluation  of  depth  charge  exercises.  It  is  suggested  that  several 
other  related  problems  be  included  in  the  formulation  of  future  plans. 
These  proposed  areas  of  investigation  are: 

1.  The  effects  of  coaching  on  team  performance  in  simu- 
lated hedgehog  attacks. 

2.  The  effectiveness  of  coaching  by  stand-by  members  of 
the  ASW  team. 
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3.  The  residual  effects  of  coaching  on  team  performance. 

4.  The  transfer  of  training  from  .attack  teacher  to  sea  ex- 
e rcises. 

The  results  of  these  investigations  should  have  considerable  prac- 
tical implications  and  contribute  valuable  information  to  the  formulation 
of  ASW  training  procedures. 
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INTRODUCTION 


Bac kg  round 

The  major  go.i  1 of  the  present  research  program  is  the  measure- 
ment of  operator  performance  in  the  ASW  weapon  system.  The  ability 
to  make  these  measurements  :s  of  practical  importance  as  well  as  of 
theoretical  interest.  If  a quantitative  measure  can  be  made  at  the 
various  operator  stations  in  a weapon  system,  it  is  possible  to  assign 
minimum  acceptable  levels  of  performance  for  the  satisfactory  em- 
ployment of  a specific  weapon.  As  measurement  techniques  are  de- 
veloped and  tested,  it  will  be  possible  to  study  available  weapon  systems 
in  this  manner.  In  time,  the  demands  placed  on  an  operator  can  be 
stated  cither  in  terms  of  a specific  weapon  or  of  any  weapon  that  may 
be  used  on  his  ship. 

The  previous  reports  of  the  work  of  the  research  program  have 
been  concerned  with  the  rationale  of  the  study  (First  Partial  Report),  a 
sea  test  of  the  procedure  (Second  Partial  Report),  and  a study  of  team 
performance  in  attack  teacher  exercises  (Third  Partial  Report).  Appen- 
dix D presents  abstracts  of  these  reports. 

Present  Problem 


During  the  analysis  of  the  data  for  the  Third  Partial  Report,  some 
of  the  results  were  of  particular  interest  to  training  activities.  The  ac- 
tivity scales  used  for  measuring  performance  could  be  used  to  predict 
attack  effectiveness.  In  addition,  the  research  team  raised  a question 
concerning  the  use  of  the  activity  scales  to  improve  team  performance. 
The  question  was  finally  stated  as  a hypothesis  to  be  tested  under  con- 
trolled conditions: 
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“Can  the  activity  scales  be  used  to  improve  ASW  team  per- 
formance by  coaching  the  team  members  during  the  practice 
attack?” 

Included  in  this  hypothesis  are  subsidi  * • y problems  which  should 
be  investigated  at  a later  date.  Among  these  are  the  time  lapse  be- 
tween coaching  and  a change  in  performance,  the  elimination  of  Class 
B attacks,  the  residual  effects  of  coaching,  the  transfer  of  training 
from  attack  teacher  exercises  to  sea  exercises,  etc.  Class  B .attacks 
in  this  and  other  Newport  studies  are  defined  as  attack  errors  of  110 
yards  or  more.  The  precise  value  of  £10  yards  is  a result  of  the  pro- 
jection scale  in  the  attack  teacher  and  the  measurement  of  error  to 
the  nearest  tenth  of  an  inch.  (This  value  is  a reasonable  appro^ma- 
tion  of  the  SurASDevDet  limit,  “exceeding  £00  yards.  ”)  The  value  for 
Class  B attacks  will  change  slightly  as  different  attack  teachers  and 
projection  scales  are  used. 

These  problems  are  not  covered  in  the  present  study.  Major  em- 
phasis has  been  given  to  the  investigation  of  the  effect  of  coaching  on 
attack  effectiveness  and  the  relative  frequency  of  lethal  attacks  (i.e., 
attacks  with  an  attack  error  of  50  yards  or  less).  Positive  results  in 
reducing  the  attack  error  and  increasing  the  proportion  of  lethal  attacks 
indicate  the  need  for  study  of  related  problems. 

Coaching  Activities 


The  importance  of  coaching  members  of  the  ASW  team  was  appar- 
ent early  in  the  research  program.  During  the  development  of  t'.ie  ac- 
tivity scales,'  emphasis  was  placed  on  the  coaching  activities  of  the 
several  officers  in  fhe  team.  The  commanding  officer  devoted  a major 
portion  of  his  time  to  advising  the  Evaluator  and  ASW  Officer.  In  turn, 
these  officers  coached  their  subordinates.  However,  none  of  the  activ- 
ity scales  measuring  coaching  activities  could  be  used  to  predict  attack 
effectivenes  s. 


'CAMPBELL,  J.  W.  , DORNY,  L.  R.  & CHANNELL,  R.  C.  The  eval- 
uation  of  ASW  team  performance  by  measurement  of  operator  activities. 
First  Partial  Report,  ONR  Contract  Nonr-35  1 ( 0 Z ) , Technical  Report 
DA  30-1,  1 May  1952,  CONFIDENTIAL. 
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Additional  experience  with  the  ASW  team  during  sea  and  attack 
teacher  exercises  led  to  the  development  of  an  operational  definition  of 
coaching.  ^ It  became  apparent  that  coaching  activities  were  not  being 
performed  adequately  for  several  reasons.  Perhaps  the  most  impor- 
tant reason  was  the  a nount  of  attention  an  officer  h.  to  devote  to  his 
primary  activities  during  the  attack- -the  conning  of  the  ship,  weapon 
selection,  attack  dec  i s ions  ,and  the  like.  There  were  also  the  problems 
of  knowing  when  coaching  was  required,  the  aspects  of  an  operator's 
performance  that  required  coaching  and  the  fact  that  the  directions  given 
by  officer  personnel  are  often  more  likely  to  be  in  general  terms  rather 
than  in  the  form  of  specific  instructions.  It  would  seem  that  a solution 
to  these  problems  could  be  found  by  delegating  coaching  responsibilities 
to  enlisted  men  whose  primary  duty  would  be  to  fulfill  the  role  of  a 
coach. 

The  Second  Partial  Report  includes  a theoretical  discussion  of  four 
general  specifications  for  coaching  an  operator: 

1.  The  supervisor  must  be  able  to  recognize  the  overt  be- 
havior typical  of  an  adequate  performance. 

2.  Immediate  recognition  of  inadequate  performance  is 
necessary. 

3.  Coaching  should  be  in  the  form  of  specific  instructions. 

4.  The  elements  of  job  performance  most  closely  related 
to  attack  success  should  be  given  priority  in  coaching. 

The  following  steps  show  how  the  rater  could  satisfy  each  of  these 
coaching  specifications  by  observing  and  coaching  operator  performance 
with  the  aid  of  activity  scales: 


'CAMPBELL,  J.  W.  , TOLCOTT,  M.  A.,  MUCHMORE,  S.  C.  , 
KELLNER,  A.  D.  I CHANNEL!.,  R.  C.  The  evaluation  of  ASW 
team  performance  by  measurement  of  operator  activities.  Second 
Partial  Report,  ONR  Contract  Non  r - 3 5 1 ( 02),  Technical  Report 
DA  30-2,  1 July  1952,  CONFIDENTIAL. 
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1.  The  observer  is  aware  of  the  overt  behavior  of  the  op- 
erator at  all  stages  of  the  attack. 

1.  The  steps  of  the  activity  scale  describe  operato  r- 
formancc  in  varying  degrees  of  “goodness.  ’’ 

3.  The  observer  could  tell  the  operator  what  he  should  do 
to  improve  his  performance  since  the  scales  are  in  be- 
havioral terms. 

4.  The  coaching  can  he  limited  to  those  activities  that  serve 
to  predict  the  attack  error. 

The  present  study  was  designed  to  test  the  effectiveness  of  coaching 
(measured  by  reduction  of  mean  attack  error)  when  the  activity  scales 
are  used  as  coaching  aids. 
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CHAPTER  / 
PROCEDURE 


As  in  previous  studies  conducted  at  Newport,  certain  modifications 
of  the  attack  teacher  equipment  and  training  procedures  were  made  to 
insure  the  collection  of  data  under  expo  rimentally  controlled  conditions. 
However,  the  practicality  and  adaptability  of  the  procedure  were  kept 
in  mind.  If  Navy  personnel  are  to  make  use  of  the  methods  developed 
in  the  course  of  this  research,  it  is  necessary  to  design  procedures 
which  are  practical  and  readily  adaptable  by  personnel  of  the  fleet. 

Selection  and  Training  of  Coaches 


In  designing  the  experimental  procedure  for  the  present  study  it 
was  decided  that  coaching  activities  within  the  ASW  team  could  be  del- 
egated to  enlisted  men  of  the  fleet.  The  three  coaches  in  this  study 
were  chosen  by  the  research  team  from  a group  of  12  enlisted  men  se- 
lected at  random  from  the  fleet  by  ComDesLant.  The  coaches  consisted 
of  two  rated  sonarmen  and  a rated  radarman.  The  coaches  had  no  pre- 
vious specific  training  or  experience  in  coaching  activities,  and  had 
approximately  the  same  background  with  respect  to  education  and  Naval 
experience.  These  men  should  be  typical  of  other  enlisted  men  in  the 
fleet  who  might  be  called  upon  to  perform  such  activities  in  future  ap- 
plications of  this  procedure. 

The  coaches  were  given  instructions  in  standard  coaching  proce- 
dures, and  were  instructed  in  the  use  of  the  scales  as  coaching  aids  by 
members  of  the  research  team.  The  course  of  instructions  covered  a 
one-day  period,  and  could  readily  be  adapted  for  use  by  the  fleet. 

Experimental  Design 

The  sample  for  this  study  consisted  of  10  ships  of  the  DD  type  se- 
lected at  random  by  the  Operations  Officer  of  DesFlot  Six.  It  was  nec- 
essary to  divide  the  ships  into  two  groups  since  the  goal  of  the  present 
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study  was  to  determine  t lie  effects  of  c oaching  as  a method  of  reducing 
the  mean  attack  error:  an  experimental  group  subjected  to  coaching 

and  a control  or  practice  group  with  no  coaching.  Each  ship  was  assigned 
to  the  attack  teacher  for  a two-day  cycle.  The  first  half-day  of  each  cycle 
consisted  of  regular  practice  runs  (P())  for  ships  in  both  the  experimental 
and  control  groups. 

This  provided  a reference  for  their  t y p i c: a 1 performance  and  gave  the 
ships  of  both  groups  an  opportunity  to  coordinate  their  activities  and  to 
reach  a relatively  stable  level  of  performance.  During  the  retmainder  of 
the  two-day  cycle,  the  experimental  group  was  coached  during  exercises, 
and  the  control  group  conducted  standard  practice  attacks  without  coach- 
ing. The  experimental  plan  is  shown  in  Table  I.  * 


Tabic  l 

Experimental  plan 


Cycle 

First 

day 

Second  day 

A.  M. 

P.  M. 

A.M.  P.M 

Experimental  group  (Coached) 

P 

o 

C1 

c,  c3 

Control  group  (Practice) 

Po 

P1 

P2  P3 

P - 
C 

P ract  ice 
Coached  . 

attacks 

attacks 

Experimental  controls  were  used  throughout  the  exercises  to  insure 
both  the  experimental  and  control  ships  operating  under  identical  condi- 
tions insofar  as  possible.  Throughout  all  cycles  the  ships  conducted 


1 

The  number  of  coaching  and  practice  sessions  selected  for  a cycle  was 
arbitrary  since  prior  to  this  study  it  was  not  possible  to  determine  the 
number  of  sessions  required  before  coaching  w'ould  be  effective. 
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standard  two-ship  coordinated  exercises  using  their  regularly  assigned 
GQ  teams,  A continuous  target  course,  consisting  of  five  standard  runs 
combined  randomly,  was  used  for  all  exercises.  This  assured  the  same 
target  maneuvers  for  all  ships.  The  time  intervals  between  changes  in 
course  and  speed  were  controlled  by  the  target  helmsman  using  a stop 
watch.  The  ships  were  alternated  on  the  basis  of  their  initial  perform- 
ance between  the  coached  and  practice  side  of  the  attack  teacher  at  the 
tnd  of  their  first  practice  session  (PQ).  This  permitted  the  equating  of 
the  experimental  and  control  groups. 

Range  and  bearing  information  for  the  CIC  spaces  was  obtained  from 
a grid  pattern  projected  on  the  plotting  screen.  In  previous  attack  teacher 
work  it  was  found  that  this  method  would  furnish  more  frequent  and  accu- 
rate information  to  CIC  as  well  as  relieve  the  behind  screen  plotters  of 
the  extra  burden  of  making  direct  measurements  from  the  screen.  In 
this  way  a possible  source  of  experimental  error  was  eliminated.  A dis- 
cussion of  this  projection  system  is  presented  in  Appendix  A. 

All  ships  of  the  coached  group  were  assigned  to  one  side  of  the  attack 
teacher  for  all  cycles.  This  was  necessary  since  a Course-to-Steer  In- 
dicator, which  is  installed  on  only  one  side  of  the  attack  teacher  at  New- 
port, is  of  importance  in  the  evaluation  and  coaching  of  the  helmsman. 

The  coaches  for  the  experimental  trials  were  assigned  to  one  side  of 
the  attack  teacher.  The  two  sonarmen  were  assigned  to  the  Bridge  and 
Sonar  compartments;  the  radarman  to  CIC.  The  stations  and  activities 
coached  are  shown  in  Figure  1. 

Inasmuch  as  the  present  study  was  also  designed  to  evaluate  team 
performance  using  measurements  of  the  attack  error  and  the  activity  rating 
scales,  operator  activities  of  both  the  coached  and  practice  exercises  were 
rated  by  Navy  personnel  with  the  scales  developed  and  used  in  previous 
studies.  '•  2 The  raters,  one  for  each  compartment,  were  selected  and 


'CAMPBELL,  J.  W.  , DORNY,  L.  C.  & CHANNELL,  R.  C.  The  eval- 
uation  of  ASW  team  performance  by  measurement  of  operator  activities. 
First  Partial  Report,  ONR  Contract  Nonr- 35 1(02),  Technical  Report 
DA  30-2,  1 July  1952,  CONFIDENTIAL. 

2MUCHMORE,  S.  C.,  KELLNER,  A.  D.  , KOPONEN.  A.  & CHANNELL, 
R.  C.  The  evaluation  of  ASW  team  performance  by  measurement  of  op- 
erator activities  . Third  Partial  Report,  ONR  Contract  Nonr  - 3 5 1(  0 2), 
Technical  Report  DA  30-3,  1 November  1952,  CONFIDENTIAL. 
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and  trained  by  the  research  team.  The  operators  whose  activities  were 
rated  included:  * 

Bridge  - Helmsman 

Sonar  Control  - ASW  Officer,  QHB  Operator, 

Range  Recorder  Operator 

C1C  - Evaluator,  C1C  Officer, 

P 1 ot  t e r 1 and  11 

The  raters  alternated  between  the  coached  and  practice  ships  with  each 
new  cycle.  This  provided  a control  so  that  no  one  rater  would  rate  the 
activities  of  the  coached  group  for  the  entire  study. 

The  practice  sessions  consisted  of  approximately  40  minute  exer- 
cises during  which  three  or  four  attacks  could  be  observed  for  each  ship. 
As  in  previous  studies,  target  and  ship  information  were  plotted  on  DRT 
paper  placed  on  the  back  of  the  projection  screen.  This  provided  a per- 
manent plot  from  which  more  accurate  measurements  of  the  attack  error 
could  be  made. 

The  attack  error  was  measured  after  each  exercise  by  members  of 
the  research  team.  Measurements  were  made  from  the  center  of  the 
weapon  pattern  to  the  center  of  the  target  position  after  allowing  sinking 
time  for  the  depth  charge.  All  measurements  were  made  to  the  nearest 
tenth  of  an  inch. 


A complete  list  of  the  activities  observed  is  shown  in  Appendix  B. 
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CHAPTER  II 


RESULTS 


* 


i 


i 


The  results  of  the  study  of  coaching  on  ASW  team  effectiveness  are 
presented  in  this  chapter.  In  general,  the  findings  indicate  an  improve- 
ment in  performance  for  those  teams  which  are  coached  in  their  activ- 
ities. This  improvement  is  significant  when  compared  to  the  perform- 
ance of  a control  group  which  carried  out  standard  attack  teacher  runs 
without  coaching  except  that  normally  done  within  the  team.  The  results 
are  presented  without  any  extensive  discussion  of  significance  and  other 
statistical  concepts  involved.  A more  technical  discussion  is  presented 
in  Appendix  C. 

Sample 

A total  of  356  attacks  were  observed  at  Newport  (Table  II).  It  was 
necessary  to  exclude  certain  runs  in  the  final  analysis  of  the  data  because 

Table  II 

Breakdown  of  attacks  observed  at  Newport 

i ■ ■ ■■ 1—  l.  i 

Runs  excluded  from  analysis 


Incomplete  cycles  90 

Class  B attacks  1 3 

Casualties  in  system  1 1 

Total  114 

Runs  analyzed 

Control  group  128 

Experimental  group  114 

Total  242 
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of  the  methods  used  in  analyzing  the  data  and  the  nature  of  the  experi- 
mental design.  Runs  during  which  equipment  casualties  occurred,  Class 
B attacks  (i.e.,  those  with  attack  errors  of  more  than  .£10  yards),  and 
runs  from  incomplete  exercise  cycles  in  which  a ship  did  not  complete 
two  full  days  of  exercises  were  excluded.  The  last  requirement  was  im- 
posed in  the  selection  of  the  final  sample  since  in  comparing  the  control 
with  the  experimental  group  it  was  necessary  for  ships  of  both  groups  to 
have  completed  the  full  cycle  of  runs.  The  final  sample  for  analysis  con- 
tained 242  runs:  1 2K  for  the  control  group  and  114  for  the  experimental 

group. 

Equating  of  Experimental  Groups 


The  manner  in  which  the  ships  were  divided  into  an  experimental 
group  and  a control  group  was  described  in  the  preceding  chapter.  The 
assignment  of  ships  was  made  in  such  a way  as  to  equate  the  groups  on 
the  basis  of  initial  performance.  By  using  this  procedure  it  was  possible 
to  work  w'ith  two  groups  which  were  approximately  equal  before  introduc- 
tion of  the  experimental  conditions.  Actually,  the  groups  differed  by 
only  5 yards  in  mean  attack  error  for  the  equating  sessions. 

Figure  2 presents  an  ogive  showing  the  cumulative  percentage  of 
attack  errors  for  each  group  during  the  equating  sessions.  An  ogive  is 
a cumulative  frequency  curve;  at  any  point  on  the  curve  it  is  possible  to 
determine  the  proportion  of  observations  that  are  as  good  as,  or  better 
than,  a given  value.  Thus,  in  the  present  curve,  we  can  determine  the 
relative  frequency  of  attacks  that  fall  within  a given  radial  attack  error. 
For  example,  the  control  group  has  22%  of  its  attacks  within  50  yards  of 
the  target  and  70%  within  100  yards.  When  the  performance  of  two  groups 
is  displayed,  the  vertical  distance  between  the  two  curves  is  the  important 
measure  and  provides  a ready  comparison  of  the  groups. 

In  preparing  curves  of  this  type,  it  is  desirable  to  "smooth”  the  fluc- 
tuations in  observed  values,  assuming  that  such  fluctuations  are  due  to 
chance.  Therefore,  there  will  not  be  complete  agreement  between  values 
taken  from  the  figures  and  comparable  values  reported  in  the  text.  The 
text  will  use  the  actual  values  observed  in  the  experimental  study. 
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SO  OO  ISO  200 


ATTACK  ERROR  < YARDS  J 


NOTE:  It  is  frequently  desirable  to  categorize 

a series  of  attacks  by  a weapon  system  into  cer- 
tain hits,  possible  hits  and  misses.  In  these 
groups,  the  kill  probability  of  “certain  hits”  is 
practically  1.00.  In  the  “possible  hit”  zone, 
the  kill  probability  dec  reases  rapidly  from  about 
1.00  to  almost  zero.  Any  kills  in  the  “miss” 
area  are  chance  occurrences  and  probably  the 
result  of  wild  rounds.  The  present  studyof  the 
depth  charge  weapon  system  defines  thesethree 
zones  as: 


Lethal  attacks  = certain  hits  - -attack  error s 
of  50  yards  or  less 

Possible  hits  - attack  errors  between  51 

and  100  yards 

Misses  = attack  errors  between  101 

and  209  yards 

Class  B attacks  = attack  errors  of  210  yards 
or  more 

Non-firing  attacks  = attacks  belayed,  weapons 
not  fired 


Figure  2.  Cumulative  percentages  of  attack  errors  during  equating 
sessions  for  experimental  and  control  groups  (smoothed 
curves ). 

In  Figure  2 it  may  be  noted  that  there  are  only  minor  differences 
between  the  groups.  The  experimental  group  has  a somewhat  higher 
percentage  of  attacks  within  the  50  yard  range  while  the  control  group 
has  a higher  percentage  of  attacks  within  the  100  yard  range.  The 
crossing-over  of  the  two  curves  indicates  the  over-all  equality  of  the 
groups  in  ASW  attack  proficiency  as  measured  by  the  attack  error. 

Relative  Attack  Performance  of  Ships 

It  has  already  been  shown  (Figure  2)  that  there  were  no  real  differ- 
ences in  the  original  performance  of  the  two  groups  of  ships.  If  these 
ships  are  arranged  in  the  order  of  their  mean  attack  errors,  it  is  pos- 
sible to  study  their  relative  effectiveness  in  attack  teacher  exercises, 
(Such  comparisons  are  routine  in  many  training  situations.)  A similar 
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ranking  of  the  ships  was  made  at  the  end  of  the  experimental  period  to 
provide  an  index  of  the  relative  performance  of  the  coached  and  control 
ships  (Table  III). 


Table  III 

Ranking  of  ships  in  order 
of  mean  attack  error 


Ship 

Equating  sc 

Practice  or 
c oa  c hed 

s sion 

Attack 
error  (yds) 

Ship 

Experimental 

Practice  or 
coached 

pe  riod 

Attack 
error  (yds) 

A 

P ractice 

60 

B 

Coac  hed 

63 

B 

Coac  hed 

75 

D 

Coached 

8 2 

E 

P ractice 

80 

F 

Coached 

84 

F 

Coac  hed 

86 

A 

Practice 

85 

D 

Coac  hed 

107 

E 

Practice 

88 

C 

P ractice 

1 1 5 

C 

Practice 

106 

An  examination  of  the  relative  ranks  of  the  ships  in  the  equating 
sessions  indicates  no  differences  between  the  coached  and  the  practice 
ships.  However,  there  are  consistent  differences  after  the  experimental 
period:  all  of  the  coached  ships  are  better  than  any  of  the  practice  ships. 

The  individual  performances  are  also  consistent.  Each  of  the  coached 
ships  does  better  in  the  experimental  period,  but  only  one  of  the  practice 
ships  shows  an  improvement. 

Effects  of  Coaching  on  Operator  Performance 


Any  change  in  ASW  team  performance  which  might  be  brought  about 
by  coaching  must  first  be  reflected  in  an  improvement  in  individual  per- 
formance. In  this  case  such  performance  was  measured  by  ratings  made 
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with  the  activity  scales.  It  was  found  that  these  ratings  improved  con- 
sistently for  the  experimental  group.  The  ratings  for  the  control  group 
showed  no  consistent  trend.  It  will  be  noted  later  that  the  fluctuations 
for  the  control  group  parallel  the  session-to- session  changes  in  mean 
attack  error.  Figure  3 points  out  the  session-to-session  change  in 
ratings  for  the  coached  and  control  groups. 


Figure  3.  Mean  activity  ratings,  as  measured  by 
activity  scales,  for  equating  and  expe r- 
imental  exercises. 
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Team  Performance  During  Experimental  Sessions 

The  change  in  individual  performance  which  is  indicated  in  the 
ratings  of  operator  activities  is  also  observable  in  team  performance 
as  measured  by  the  attack  error.  Cumulative  percentage  curves  for 
the  experimental  sessions  ai>  shown  in  Figure  4. 


lethal 

ATTACKS 


POSSIBLE 

HITS 


EXPERIMENTAL 
GROUP  V 

I 


77 

/v 


CONTROL 

GROUP 


50  100  150  200 

ATTACK  ERROR  ( YARDS ) 


Figure  4.  Cumulative  percentages  ol  attack  errors 
during  experimental  sessions  for  exper- 
imental and  control  groups  (smoothed 
curves ). 
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The  improvement  of  the  coached  group  is  apparent  in  comparing 
these  curves  with  the  curves  obtained  from  the  equating  sessions  (Fig- 
ure 1).  Where  the  two  groups  were  essentially  the  same  in  perform- 
ance during  the  equating  sessions,  the  coached  group  shows  much  better 
performance  during  the  experimental  sessions. 

Lethal  Attacks 

# 

The  distribution  of  attacks  for  the  coached  group  was  found  to  have 
shifted  toward  the  lower  end  of  the  attack  error  scale.  This  shift  is 
apparent  in  the  increase  in  proportion  of  lethal  and  “possible  hit**  attacks. 
Although  the  proportion  of  good  attacks  tended  to  decrease  for  the  control 
group  during  the  experimental  period,  the  shipswhich  were  coached  dur- 
ing their  attacks  actually  increased  their  proportion  of  good  attacks. 

These  changes  in  proportions  are  shown  graphically  in  Figure  5. 

The  control  and  experimental  groups  differed  somewhat  in  percent 
of  lethal  attacks  during  the  equating  runs  (9%).  This  difference  is  not 
statistically  significant.  * However,  the  difference  in  proportion  of  lethal 
attacks  during  the  experimental  sessions  was  significant  at  the  1%  level 
of  confidence.  The  coached  group  changed  from  2. 8%  to  35%  in  proportion 
of  lethal  attacks.  During  the  same  sessions,  the  group  which  used  stand- 
ard attack  procedures  without  coaching  went  from  19%  to  17%  in  propor- 
tion of  lethal  attacks. 

In  terms  of  possible  hits,  the  control  group  was  16%  superior  at  the 
outset.  During  the  experimental  sessions  this  difference  was  reduced  to 
make  the  groups  approximately  equal.  The  proportion  of  misses  in- 
creased for  the  control  group  while  the  experimental  group  reduced  their 
proportion  of  misses  by  13%. 


^This  difference  cannot  be  considered  as  a real  difference  since  in  re- 
peated experiments  of  this  nature  a difference  this  large  would  occur 
by  chance  too  frequently.  We  can  thus  assume  that  no  real  differences 
exist.  When  the  significance  levels  (1%  level,  5%  level)  are  noted  in 
this  report  these  figures  are  the  probabilities  that  such  differences 
could  be  attributed  to  chance. 
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Comparison  of  Attack  Error  Means 


The  mean  attack  errors  for  the  two  groups  are  compared  in  Table 
IV,  first  during  the  equating  trials  and  again  for  the  experimental  ses- 
sions. 

Table  IV 

Mean  performance  of 
experimental  and  control  group a 


Mean  Performance  ( in  yards  ) 


Initial  runs 

Experimental  runs 

Change 

Control  group 

81 

92 

+ 11 

Experimental  group 

86 

75 

- 11 

The  groups  differed  by  5 yards  in  mean  performance  during  the 
equating  trials.  This  difference  is  not  significant.  On  the  other  hand, 
the  difference  between  the  groups  during  the  experimental  sessions  was 
17  yards.  This  difference  is  large  enough  to  be  expected  to  occur  by 
chance,  or  sampling  fluctuation,  less  than  1 time  in  100  (significant  at 
the  1%  level  of  confidence). 

Fluctuations  in  Performance  of  Control  and  Experimental  Groups 


The  change  in  performance  noted  from  the  equatirrg  to  the  experi- 
mental sessions  is  also  of  interest.  The  coached  group  improved  11 
yards  while  at  the  same  time  the  control  group  went  down  in  perform- 
ance to  the  extent  of  1 1 yards. 
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The  improvement  of  the  experimental  group  is  presumably  a result 
of  the  coaching.  However,  the  reason  for  the  decline  in  performance 
for  the  control  group  is  not  immediately  obvious.  Plotting  the  session- 
to-session  attack  error  means  for  the  control  and  experimental  groups 
shows  a general  improvement  for  the  coached  group  with  a slight  fall- 
off  in  the  final  session.  The  performance  of  the  control  group  appears 
to  fluctuate  more  or  less  at  random  (Figure  6). 


tr 

o 

£T 
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o 
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I 

< 


I 2 3 
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Figure  6.  Mean  attack  error  for  experimental  and  control 
groups. 


It  is  possible  that  the  fall-off  in  performance  for  the  control  group 
during  the  afternoon  sessions  is  an  indication  of  a diurnal  cycle  of  team 
performance.  Such  factors  as  fatigue  or  boredom  may  be  at  work  to 
bring  about  a decrease  in  performance  during  these  sessions.  It  will 
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be  possible  to  investigate  this  fluctuation  in  future  attack  teacher  studies 
to  determine  whether  it  is  a typical  diurnal  fluctuation  or  merely  a 
chance  occurrence.  Analysis  of  the  data  with  this  objective  in  mind  can 
be  made  without  any  change  in  proposed  experimental  design.  Further 
study  of  ASW  attack  effectiveness  should  be  made  before  any  conclusions 
are  drawn  regarding  the  observed  fluctuation. 

Validity  of  Activity  Scales 

It  was  mentioned  earlier  that  the  ratings  of  the  control  group  paral- 
leled the  attack  error  means  from  s cs  sion  - to- s e s s ion  (Figures  3 and  6). 
This  agreement  might  be  considered  an  indication  of  the  relationship  be- 
tween the  ratings  and  the  attack  error.  Calculation  of  a correlation  co- 
efficient for  these  variables  has  shown  a significant  relationship  to  exist. 

Validations  of  the  activity  scales  in  the  previous  studies  (Third  Par- 
tial Report)  have  shown  that  the  attack  error  could  be  predicted  with  a 
fair  degree  of  accuracy  by  the  use  of  the  scales  in  combination.  It  was 
further  reported  that  a satisfactory  prediction  could  be  made  by  simply 
adding  all  the  ratings  to  get  a total  measure  of  team  performance.  The 
correlation  coefficient  obtained  in  this  manner  was  almost  as  high  as  the 
multiple  R obtained  using  more  complicated  beta  weights. 

A similar  procedure  was  followed  with  the  ratings  obtained  in  this 
study  using  the  same  predictors  as  previously.  A correlation  of  . 34 
was  obtained.  The  previous  study  resulted  in  a validity  estimate  of 
r = . 42  based  on  a sample  of  81  observations.  The  present  estimate, 
r ~ .34  (n  - 1 28)  does  not  differ  appreciably  from  this  original  value. 

In  fact,  a test  of  the  difference  indicates  that  we  would  expect  suchdis- 
crepencies  about  half  the  time  simply  as  a result  of  sampling  errors. 
Thus,  the  rating  scales  continue  to  show  a significant  positive  relation- 
ship with  the  attack  error  criterion. 

The  ratings  obtained  in  this  study,  as  in  the  previous  Newport  study, 
were  made  by  enlisted  Naval  personnel.  The  scales  were  used  by  these 


Using  the  conventional  formula  for  testing  the  significance  of  differences 
between  correlations  results  in  a critical  ratio  of  .67. 
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men  after  a brief  orientation  period  with  only  a minimum  amount  of 
supervision  by  research  personnel.  Therefore,  the  obtained  results 
are  comparable  to  what  might  be  expected  from  application  in  the  fleet. 

Future  Research  Plans 

The  results  of  this  study  indicate  that,  in  general,  an  improvement 
in  team  performance  in  the  attack  teacher  can  be  expected  when  opera- 
tors of  the  ASW  team  are  coached.  To  fully  evaluate  the  effectiveness 
of  coaching  on  depth  charge  exercises  a study  of  this  nature  should  be 
repeated.  If  comparable  results  are  obtained,  we  may  then  say  with 
considerable  confidence  that  team  effectiveness  ean  be  Improved  by 
coaching  the  individual  operators. 

The  present  experiment  also  suggests  several  related  problems 
which  should  be  considered  in  the  formulation  of  plans  for  future  study 
of  the  ASW  attack  system.  An  extension  of  this  present  study  to  include 
other  weapons  in  the  ASW  system  would  provide  information  of  value  to 
the  Navy  in  the  formulation  of  training  procedures  for  anti-submarine 
warfare.  For  example,  it  would  be  desirable  to  study  the  effects  of 
coaching  on  simulated  hedgehog  attacks.  The  findings  in  the  present 
study  suggest  that  coaching  will  materially  improve  the  effectiveness 
of  a weapon  system.  If  a similar  improvement  can  be  brought  about  by 
coaching  in  the  more  advanced  weapon  systems,  a significant  improve- 
ment will  be  effected  in  over-all  ASW  proficiency.  To  study  the  effects 
of  coaching  on  weapon  systems  other  than  the  depth  charge  the  same 
procedure  and  experimental  design  can  be  used  as  employed  in  the  pres- 
ent study. 


Also,  if  the  results  of  the  present  study  can  be  confirmed  by  the 
repeat  study,  then  it  would  be  desirable  to  consider  the  use  of  stand-by 
members  of  the  ASW  team  as  coaches.  Within-team  coaching,  where 
stand-by  members  of  ASW  team  perform  coaching  duties,  provides  more 
efficient  utilization  of  existing  personnel  since  coaching  personnel  would 
not  have  to  be  drawn  from  outside  the  team.  The  results  of  such  a study 
would  be  a valuable  extension  to  the  present  work  if  changes  in  team  per- 
formance could  be  effected  by  using  team  operators  as  coaches  during 
their  relief  periods. 
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Another  problem  closely  related  to  the  present  investigation  is  the 
study  of  the  residual  effects  of  coaching  on  team  performance.  The  re- 
sults of  the  present  study  have  raised  several  interesting  questions  re- 
garding this  point.  For  example,  how  much  training  should  be  given 
teams  in  the  attack  teacher  to  maximize  learning  and  maintain  profi- 
ciency? An  experimental  study  of  this  problem  should  give  valuable  in- 
formation with  respect  to  the  spacing  of  attack  teacher  training  sessions 
and  the  frequency  with  which  re-training  should  be  given.  These  studies, 
supplemented  by  additional  research,  could  supply  answers  to  the  ques- 
tion of  when  a ship  is  ready  to  engage  in  sea  exercises.  To  make  the 
most  efficient  use  of  live  submarine  time  at  sea,  the  ASW  team  should 
have  attained  an  adequate  level  of  performance.  The  attack  teacher  can 
be  a valuable  aid  in  preparing  the  ASW  team  for  sea  exercises.  This  is 
especially  true  in  the  case  of  dual-ship  attacks  in  which  communication 
between  the  ships  plays  an  important  role. 

Future  plans  for  research  should  also  include  a study  of  the  degree 
of  transfer  of  training  from  attack  teacher  to  sea  exercises.  Previous 
experimental  studies  in  the  attack  teacher  have  furnished  a methodology 
for  evaluating  team  performance  in  shore  training.  The  attack  error 
criterion  is  considerably  less  reliable  at  sea.  However,  using  the  visual 
estimates,  with  the  refinements  which  will  have  been  implemented  at  the 
time,  an  estimation  of  the  relationship  of  attack  teacher  performance  to 
sea  performance  can  be  made.  A study  of  this  nature  should  lead  to  more 
effective  utilization  of  shore-based  training  facilities. 
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APPENDIX  A 

Obtaining  Range s and  Bearing s of  Assist 
Ship  in  the  Dual  Attack  Teacher 
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The  procedure  lor  utilization  of  the  dual  QFA  attack  teacher  in 
ASW  training  involves  obtaining  the  ranges  and  bearings  of  the  other 
ship.  This  information  is  required  to  enable  the  CIC  of  each  ship  to 
maintain  a plot  of  the  other  ship.  The  procedure  currently  in  use  re- 
quires the  use  of  two  men  behind  the  screen  whose  job  it  is  to  use  a 
measuring  arm  similar  to  the  Universal  Drafting  Arm  exc  ept  that  it 
is  designed  to  be  used  on  a vertical  surface.  The  instrument  is  pro- 
vided with  a pointer  which,  when  allowed  to  swing  freely  as  the  arm 
is  placed  on  a line  between  the  ships,  indicates  the  true  bearing  from 
one  ship  to  the  other.  The  arm  is  scaled  so  that  ranges  may  be  read 
from  it.  The  information  so  obtained  is  transmitted  through  a talker 
to  the  CIC’s. 

It  was  found  that  the  use  of  the  above  system  did  not  furnish  accurate 
nor  frequent  enough  information  to  the  ship  CIC's.  In  order  to  control 
the  experimental  conditions  to  the  greatest  extent,  an  alternative  system 
was  implemented,  one  in  which  extraneous  sources  of  error  could  be 
minimized.  The  substitute  system  involved  the  use  of  a 35  n rn  pro- 
jector which  was  used  to  project  a polar  grid  on  the  screen.  A talker 
was  stationed  at  the  projector  to  read  and  transmit  the  ranges  and  bear- 
ings to  the  CIC’s. 

By  use  of  a filter  on  the  projector  it  was  possible  to  project  the  grid 
in  red.  Projecting  the  image  in  red  provided  two  advantages:  1)  the 

short  wave  length  of  the  red  light  did  not  interfere  with  the  photo-electric 
mechanism  of  the  attack  teacher  projection  equipment,  and  2)  the  dif- 
fused red  light  which  showed  through  the  screen  did  not  affect  the  dark 
adaptation  of  the  plotters. 

For  convenience  in  operation,  the  polar  grid  projector  was  placed 
on  top  of  the  pill  projector  mount  and  was  easily  trained  on  the  appro- 
priate ship  by  manipulating  the  horizontal  and  vertical  controls  on  the 
mount.  The  operator  was  provided  with  a set  of  sound-powered  phones 
which  were  hooked  into  the  21  JS  circuit.  The  operator’s  job  in  obtain- 
ing ranges  and  bearings  and  transmitting  them  to  the  CIC’s  involved 
the  following  steps: 

1.  Turn  on  projector. 

2.  Manipulate  control  knobs  so  as  to  train  grid  projection  on  ship. 
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3.  Read  range  and  bearing  to  assist  ship  into  SP  phones. 

4.  Turn  off  projector. 

This  procedure  took  about  5 seconds  to  complete.  Ranges  and 
bearings  were  usually  sent  at  the  rate  of  1 per  minute  and  frequently 
faster.  \ degree  of  accuracy  which  was  well  within  the  required 
limits  iwis  obtained.  In  general,  the  use  of  the  new  system  provided 
the  following  advantages: 

1.  More  frequent  and  more  accurate  information  was  furnished 
to  the  CIC's  of  both  ships. 

Plotters  behind  the  projection  screen  were  free  to  plot  the 
ships  and  submarine  without  the  additional  burden  of  obtain- 
ing ranges  and  bearings.  Thus,  more  satisfactory  plots 
were  obtained. 

3.  One  communication  link  was  eliminated  by  having  the  talker 
read  the  range  and  bearing  directly  off  the  screen  into  the 
SP  phones. 

The  implementation  of  this  system  in  the  QFA  attack  teacher  exer- 
cises involved  only  minor  changes  in  over-all  operating  procedures. 
However,  throughout  the  exercises,  the  use  of  the  slide  projector  served 
to  keep  the  flow  of  range  and  bearing  information  on  a more  than  adequate 
level.  The  projector  was  removed  upon  completion  of  the  experiment, 
but  it  was  indicated  by  operating  personnel  that  the  system  would,  if  pos- 
sible, become  part  of  the  standard  operating  procedure  in  the  OFA  in- 
stallation. 
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APPENDIX  B 


Operator  A ctivities  Obterved  in  the 
Attack  Teacher 
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, SONAR  CONTROL 

ASW  Officer 

1.  Coaching  of  operators 

2.  Information  control  of  (UB  Plot  and)  sonar 
QHB  (Stack)  Operator  * 

1.  Search  procedure 

2.  Tracking  ta rget s 
Range  Recorder  Operator 

1.  Range  information 

2.  Range  rate 

CIC 

Evaluator 

Coaching  of  CIC  team 
CIC  Officer 

Communication  with  assisting  ship 
Plotte  r s 

1.  Completeness  of  plot 

2.  Neatness  of  plot 

BRIDGE 

Helmsman 

Course  at  time  of  fire 

^Searchlight  type  sonar  equipment  was  employed  at  the  Newport  attack 
teacher.  This  scale  was  adapted  for  use  with  this  equipment. 
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APPENDIX  C 


Statistical  Significance  of  Results 


( 
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The  technical  reader,  who  may  be  concerned  with  research  on  prob- 
lems similar  to  those  covered  in  this  report,  may  be  interested  in  a 
more  detailed  presentation  of  the  results  of  the  statistical  analysis. 

Other  readers,  interested  in  the  operational  or  training  implications  of 
the  results,  usually  have  little  interest  in  such  information.  This  appen- 
dix is  therefore  prepared  for  the  technical  reader  and  presupposes  a 
working  knowledge  of  statistical  methods,  information  which  most  readers 
have  had  no  occasion  to  use  since  their  undergraduate  days.  All  of  the 
information  presented  here  has  been  covered  in  Chapter  II--Results. 

The  crucial  comparison  to  be  made  in  examining  the  data  gathered 
in  this  study  is  the  difference  attributable  to  the  experimental  treatment, 
i.  e.  , coaching.  As  a preliminary  to  this,  it  must  be  demonstrated  that 
any  original  differences  between  the  groups  are  probably  chance  deter- 
mined. Variance  analysis  provides  a method  for  making  this  test. 

In  making  this  analysis,  the  variance  was  divided  into  that  portion 
accruing  from  the  groups  to  which  ships  were  assigned  (experimental  or 
control),  the  mean  performance  of  the  individual  ships,  and  the  intra- 
ship variability  of  performance  (see  Table  V). 

Table  V 

Analysis  of  variance  for  P sessions 


Between  conditions 

5 

1 

5.0 

--  (4.04,  7.  19)* 

Between  ships 

193 

5 

38.  6 

1. 80(2.41, 3.42)* 

Within  ships 

1 029 

48 

21.4 

TOTAL 

1227 

54 

^Required  F ratio  for  the  5%  and  1%  levels  of  confidence 

This  analysis  indicates  that  the  variance  in  attack  error  perform- 
ance observed  during  the  equating  session  was  caused  by  the  intra-ship 
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variability.  Thus,  we  are  able  to  conclude  that  there  were  no  essential 
differences  between  the  two  groups  of  ships  at  the  beginning  of  the  study. 

The  results  of  the  testing  of  the  difference  in  proportion  of  lethal 
attacks  during  the  equating  trials  are  shown  in  Table  VI. 

Table  VI 

Difference  between  proportions  of 
lethal  attacks  during  sessions 


Number  Lethal  Non-lethal 


Experimental  group 

29 

28% 

72% 

Cont  rol  g roup 

27 

19% 

51% 

Obtained  difference  9 % 
O prop j - prop^  — 11.2 


t — — j-  - . 80  (not  significant) 

While  there  was  a difference  of  9%  in  proportion  of  lethal  attacks 
between  the  two  groups  during  the  PQ  sessions,  this  difference  may  be 
attributed  to  chance  alone.  With  samples  of  this  size  the  possibilities 
of  obtaining  a difference  of  this  magnitude, when  no  real  difference  ex- 
ists between  the  two  populations,  are  too  great  to  accept  this  as  sig- 
nificant. 

Performance  During  Experimental  Trials 

The  analysis  of  variance  for  performance  of  the  groups  during  ex- 
perimental sessions  was  computed  in  the  same  manner  as  that  for  the 
equating  session  (see  Table  VII). 

It  may  be  noted  that  the  difference  between  the  two  groups  after 
coaching  is  significant  at  the  1%  level  of  confidence.  In  other  words, 
the  obtained  difference  between  the  coached  group  and  the  practice 
could  be  obtained  by  chance  alone  (sampling  fluctuation)  less  than  1 
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time  in  100.  Thus,  it  may  be  concluded  that,  in  this  sample,  the  coached 
ships  were  brought  to  a performance  level  which  differs  significantly 
from  that  of  the  practice  ships* 

Table  VII 


Analysis  of  variance  for  experimental  sessions 


Source 

SS  df 

MS 

F 

Between  conditions 

152  1 

152.0 

7.  07(3.90,  6.  79)* 

Between  ships 

349  5 

69.8 

3.  25  (2.  27,  3.  12)* 

Within  ships 

3863  180 

21.5 

TOTAL 

4364  186 

♦Required  F ratio  for  the  5%  and  1%  levels 

of  confidence 

In  analyzing  the  differences  between  the  groups 

in  proportion  of 

lethal  attacks,  the  folio 

wing  results 

were  obtained. 

Table 

VIII 

Difference 

between  proportions  of  lethal 

attacks 

during  experimental  sessions 

Number 

Lethal 

Non-  lethal 

Experimental  group 

103 

3 5% 

65% 

Control  group 

85 

17% 

83% 

Obtained  difference  = 1C% 

O’  propj  - prop^  = 6.3 
18 

t = — - = 2.86  (Significant  at  .01  level) 
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APPENDIX  D 


Abstracts  of  First,  Second  and  Third 


Partial  Reports 
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ABSTRACT  OF  THE  FIRST  PARTIAL  REPORT 


The  Evaluation  of  ASW  Team  Performance 
by  Measurement  of  Operator  Activities 


Rationale 


Team  activities  exist  at  three  levels:  activities  performed  by  the 
individual  operator,  the  complex  of  closely  related  activities  going  on 
in  a space  such  as  CIC  or  Sonar  Control,  and  finally  the  integration  of 
the  functions  of  the  various  spaces  into  the  performance  of  the  ship. 
Each  practice  attack  can  be  considered  as  a separate  work  sample  with 
a measurable  output,  the  attack  criterion  measure. 

The  various  activities  of  an  individual  operator,  after  quantifica- 
tion of  the  task,  can  be  Combined  in  a regression  equation  to  predict 
the  attack  criterion.  In  conventional  statistical  terminology,  this  re- 
lationship is  represented  as: 


w . 

a 1 

Pal 

* Wa2 

p 

1 a 2 * ‘ • 

••‘Wan 

P 

an 

4-  C - E 

a 

Wbl 

^1 

+ Wb2 

P 

b2  * • ’ 

— Wbn 

Pbn 

+ C - E. 

b 

w 

an 

‘ Reg 

ression 

weight  of  activity 

n for 

operator  a 

P Performance  level  of  activity  n for  operator  a 

E^  - Predicted  criterion  score  using  activities  of  operator  a 

Since  the  activities  of  various  operators  are  interrelated  in  that 
the  basic  input  data,  sonar  range  and  bearing,  are  ‘’processed”  by  the 
various  members  of  the  team,  it  is  desirable  to  use  the  performance 
of  the  operators  to  predict,  as  well  as  possible,  the  performance  of 
the  ship  attack  team.  It  is  shown  that  this  can  be  done  with  the  equation 
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W»  P*  + W » P ■'  + W>  P> W»  P»  4-  C = E 

a a bb  cc  nn 

whe  re 

W*  = Regression  weight  for  predictor  activities  of  operator  a 

cl 

P’  = Mean  performance  level  on  predictor  activities  of  oper- 
a 

ator  a 

E Predicted  criterion  score  using  activities  of  ali  operators 
Activity  Ratings 

The  problems  involved  in  the  use  of  units  peculiar  to  the  various 
activities  are  complex  and  excessively  time-consuming.  Applied  psy- 
chological research  has  made  extensive  use  of  rating  scales  since  the 
first  World  War,  with  positive  results  in  the  evaluation  of  personnel 
efficiency. 

Rating  scales  were  developed  with  the  following  requirements  ful- 
filled: 

1.  Scales  were  in  job  behavior  terms. 

2.  All  major  activities  of  an  individual  that  had  a logical  rela- 
tionship to  the  success  or  failure  of  the  attack  were  covered. 

3.  All  scales  were  five  step  scales,  with  the  middle  step  des- 
criptive of  “4.0”  performance. 

4.  Almost  identical  activities  (e.  g.  , talkers  in  various  locations) 
were  covered  by  the  same  scale. 

Criterion  Measurement 


Several  criterion  measures  of  the  effectiveness  of  team  perform- 
ance were  investigated.  These  included  the  attack  error,  bearing 
error,  location  error,  range  error,  and  duration  of  lost  contact. 
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There  are,  of  course,  many  criteria  which  can  be  used  to  measure 
the  over-all  effectiveness  of  an  ASW  team.  Among  these  uould  be 
sonar  location  error,  prediction  error,  kill  probability,  attack  error, 
etc.  However,  the  criterion  to  be  used  in  the  early  phases  of  the  pro- 
gram willbe  the  attack  error.  A technique  for  measuring  the  attack 
error  with  shipboard  equipment  is  described.  The  major  limitations 
to  this  technique  are: 

1.  Short  duration  pf  the  air  bubble  on  the  surface. 

Z.  Difficulty  in  locating  the  bubble  in  sea  states  greater  than  1 or 

under  certain  giare  conditions. 

3.  Frequently,  unfavorable  viewing  angles  (relative  bearing  of 
160  to  ZOO)  occur  due  to  standard  maneuver  plans. 

4.  Excessive  ranges  to  the  bubble  due  to  slow  rising  rate  of  bubble 
(approximately  two  feet  per  second). 

Attack  Teacher  vs.  Sea  Exercises 


The  attack  teacher,  as  with  most  synthetic  training  devices,  is  a 
convenient  means  for  training  large  numbers  of  operators.  Sea  exer- 
cises, while  far  more  costly  and  time-consuming,  are  more  nearly 
like  combat  anti-submarine  operations. 

Research  considerations  indicate  the  use  of  sea  exercises,  although 
the  criterion  measurement  at  sea  may  seriously  limit  the  results.  The 
attack  teacher,  while  facilitating  the  measurement  of  attack  errors,  is 
deficient  in  simulation  of  sea  exercises,  communication  facilities  and 
integration  of  CIC  and  the  Bridge  as  parts  of  the  attack  team. 

A series  of  attack  teacher  exercises  have  been  observed  at  the  Fleet 
Sonar  School  (San  Diego,  California),  and  certain  conclusions  are  drawn 
from  the  data. 

1.  Enlisted  students  are  not  adequate  subjects  because  of  the  in- 
fluence of  the  instructor  on  the  attack  error  under  present 
training  doctrine. 
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2.  Non-firing  attacks  must  be  excluded  from  the  data,  although 
data  can  be  gathered  as  to  causation  of  these  failures. 

3.  I wo-ship  coordinated  attacks  provide  a better  training  prob- 
lem for  C1C  than  do  single-ship  exercises. 

Analysis  of  Sea  Exercise  Data  ^ 

The  general  plans  for  collection  and  analysis  of  data  on  sea  exer- 
cises are  described.  The  plans  for  the  intensive  study  of  the  various 
activity  rating  scales  to  improve  the  prediction  are  also  covered. 

Future  Work 


The  preliminary  plans  for  future  data  collection,  including  the  sea 
exercises  covered  in  the  present  report,  are  described.  These  include 
attack  teacher  studies,  systems  research,  and  various  small  experi- 
mental studies  to  test  hypotheses  developed  by  the  research  team. 

Implications  for  Future  Research 


Potential  uses  of  the  present  methodology,  after  demonstrating  its 
validity,  are  presented  for  consideration.  Among  the  more  obvious 
uses  are  criteria  in  personnel  research  on  selection  and  training. 
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ABSTRACT  OF  THE  SECOND  PARTIAL  REPORT 


Operator  Activities  in  Relation  to  Sea  ASW  Exercises 
ASW  Training 

Current  ASW  training  exercises  were  observed  both  ashore  and  at 
sea.  Some  of  the  major  differences  between  sea  and  shore-based  train- 
ing are  presented  with  emphasis  on  the  relative  value  of  these  two  modes 
of  t raining. 

Shore-based  training:  It  has  been  demonstrated  that  knowl- 

edge of  results  of  a performed  task  facilitates  the  learn- 
ing process.  Knowledge  of  results  of  attack  effective- 
ness is  almost  immediately  available  to  crew  members 
during  shore-based  training.  The  instructor  reviews 
the  performance  of  the  team  and  provides  a critique. 

However,  motivation  is  not  as  high  as  during  sea  exer- 
cises due  to  the  lack  of  competition  and  realism  in  the 
synthetic  problems. 

Sea  exercises:  Typical  sea  exercises  do  not  provide  a re- 

view of  team  performance  and  precise  data  on  target 
course  and  attack  effectiveness  are  not  usually  avail- 
able to  the  crew.  When  the  knowledge  of  results  of  the 
attack  are  communicated  to  the  crew,  the  accuracy  of 
the  information  is  open  to  question  because  of  the  guess- 
work involved  in  the  estimates  of  the  attack  error.  Mo- 
tivation is  high  because  of  the  realistic  and  challenging 
nature  of  the  problem. 

The  sea  exercises  used  for  experimental  purposes  were  observed 
at  Guantanamo  Bay.  A total  of  356  attacks  were  observed  of  which  117 
attacks  were  used  to  obtain  preliminary  information  that  would  serve 
as  a basis  for  eliminating  certain  activity  rating  scales.  This  provided 
a total  of  239  attacks  which  were  used  to  obtain  rating  data  on  the  se- 
lected activity  scales.  From  this  group  of  239  attacks,  139  attacks  were 
rejected  for  various  reasons.  The  final  sample  for  analysis  consisted 
of  100  attacks,  an  attrition  of  about  58%. 
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General  Quarters  Job  Descriptions 


The  activity  rating  scales  had  been  developed  after  a survey  of  the 
available  material  on  ASW  training,  including  a study  of  the  training 
activities  of  the  Fleet  Training  Center,  Norfolk.  Observations  of  the 
ASW  team  during  sea  exercises  at  Guantanamo  Bay  provided  data  on 
the  activity  ratings  and  job  duties  of  the  specific  members.  These  data 
also  provided  information  of  value  in  determining  some  of  the  causes  of 
non-firing  attacks  and  differences  in  operating  procedures.  Some  of 
the  outstanding  factors  which  affect  operating  procedures  include  atti- 
tudes and  training  of  Commanding  Officers,  capabilities  of  personnel, 
training  stations  at  which  procedures  were  learned,  equipment  arrange- 
ment and  type  of  exercise. 

Attack  Error  As  A Measure  of  Attack  Effectiveness 

The  development  of  an  adequate  criterion  measure  of  team  per- 
formance has  been  one  of  the  major  problems  in  this  phase  of  the  re- 
search program.  The  criteria  which  have  been  investigated  include 
location  error,  bearing  error,  duration  of  lost  contact,  probability  of 
kill,  and  the  attack  error.  Research  personnel  have  been  working  on 
the  development  of  a satisfactory  attack  error  measurement  device  for 
a number  of  years,  but  a practical  (and  inexpensive)  method  is  still  not 
available.  During  the  experimental  phase  of  the  present  research  pro- 
gram, several  measures  of  the  effectiveness  and  limitations  of  team 
performance  were  investigated.  These  are  discussed  along  with  sug- 
gestions for  new  or  improved  measures.  Because  of  the  lack  of  a more 
precise  measure,  visual  estimates  of  the  attack  error  are  used  in  this 
present  study  as  the  criterion  measure. 

Results  of  Sea  Observations 


An  analysis  of  the  results  of  this  study  indicates  that  there  is  gen- 
eral agreement  between  the  research  findings  and  the  results  of  a log- 
ical evaluation  of  the  important  activities  of  the  ASW  system.  The  im- 
mediate usefulness  and  application  of  these  findings  is,  however,  limited 
because  of  the  lack  of  a more  precise  criterion  measure.  The  relation- 
ship between  the  present  criterion  estimate  and  the  critical  activities  of 
the  ASW  team  indicates  that  the  rating  technique  is  a potentially  valuable 
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method  for  measuring  operator  and  team  performance  and  thus  the 
methodology  used  in  this  study  may  be  profitably  applied  to  other  re- 
search problems  on  team  performance. 

Related  Research  Problems 


During  the  field  trips  the  research  team  observed  variations  in 
attack  procedures  which  suggest  several  training  research  problems. 
From  an  investigation  of  these  areas  it  may  be  possible  to  determine 
those  procedures  which  will  provide  better  and  more  effective  team 
performance.  A study  of  ASW  activities  also  revealed  that  it  would  be 
profitable  to  conduct  a human  engineering  analysis  of  the  arrangement 
of  existing  ASW  equipment.  This  is  particularly  important  since  it  is 
felt  that  working  conditions  within  ASW  spaces  could  be  improved  by 
making  relatively  minor  modifications  of  equipment  arrangement  while 
ships  are  undergoing  a periodic  overhaul.  A discussion  of  some  of 
these  human  engineering  recommendations  are  presented  in  this  report. 

Future  Research  Studies 


Future  plans  for  the  research  program  include  further  studies  of 
attack  teacher  exercises  and  additional  sea  observations.  The  plans 
call  for  further  investigation  of  the  validity  and  reliability  of  the  activ- 
ity scales  and  the  experimental  testing  of  various  hypotheses. 
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ABSTRACT  OF  THE  THIRD  PARTIAL  REPORT 


Operator  Activities  in  Attack  Teacher  ASW  Exercises 
ASW  Training 

Incidental  to  the  research,  the  report  includes  observations  of  the 
attack  teacher  training  methods.  Motivation  should  be  increased  by 
critiques,  orientation  lectures  and  the  use  of  more  realistic  and  com- 
plex target  courses.  The  present  emphasis  in  training  appears  to  be 
placed  on  learning  by  repetition.  However,  instruction  and  coaching 
to  be  effective  should  point  out  poor  performance  as  well  as  adequate 
performance  as  soon  as  it  occurs.  The  criticisms  should  be  specific 
and  where  possible  the  operators  should  have  immediate  knowledge  of 
results.  Coordination  within  teams  and  between  ships  is  necessaryfor 
the  development  of  effective  ASW  team  performance. 

Procedure 


Regular  dual-ship  attack  teacher  exercises  were  conducted  for  ships’ 
teams.  The  target  course  of  the  teams  consisted  of  standard  runs  com- 
bining both  full  and  limited  evasive  target  maneuvers.  The  activities  of 
the  ASW  Officer,  QHB  Operator,  Range  Recorder  Operator,  Evaluator, 
CIC  Officer,  Plotters  and  Helmsman  were  rated  during  the  trials  by  Navy 
personnel  familiar  with  the  activities.  Permanent  plots  were  recorded 
for  each  trial  and  the  attack  error  was  measured  and  later  corrected  to 
account  for  target  evasion  after  loss  of  contact. 

Results 


1.  The  data  of  81  attacks  were  used  in  tabulating  the  results.  Class 
B,  single-ship  and  attacks  without  sufficient  data  were  excluded  from  the 
analysis. 

2.  Measures  of  the  attack  error  criterion  were  adjusted  to  allow  for 
random  turns  of  the  target  after  loss  of  contact. 

3.  rtn  analysis  of  the  data  indicated  a homogeneous  group  of  ships’ 
teams  which  justifies  combining  them  in  various  ways  for  purposes  of 
investigation. 
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4.  An  evaluation  of  target  aspect  showed  that  there  was  no  sig- 
nificant difference  in  attack  error  depending  on  the  target  angle  (bow, 
stern  or  beam  aspects)  during  the  final  stage  of  attack. 

5.  The  QHB  Operator’s  and  the  ASW  Officer’s  composite  rating 
scales  correlated  most  highly  with  the  attack  error  criterion. 

6.  Correlations  between  the  criterion  and  the  rating  scores  were 
higher  in  the  attack  teacher  than  the  correlations  obtained  in  sea  exer- 
cises which  may  be  due  to  the  more  reliable  measure  of  attack  error  in 
the  attack  teacher. 

7.  The  correlations  between  the  criterion  and  the  rating  scores 
tended  to  maintain  the  same  positions  of  rank  according  to  magnitude 
in  both  the  sea  exercises  and  in  the  attack  teacher. 

8.  Team  members  who  performed  well  in  one  activity  tended  to 
perform  well  in  others. 

9.  Individual  team  members  who  performed  well  were  usually 
members  of  teams  in  which  others  performed  well. 

1 0.  When  all  of  the  ratings  were  combined  and  correlated  with  the 
attack  error  criterion,  the  relationship  compared  favorably  with  those 
usually  found  in  rating  techniques. 

1 1.  Ships’  teams  rated  high  on  the  activity  rating  scales  have  a 
larger  percentage  of  lethal  attacks  than  ships’ teams  rated  low. 

12.  A small  improvement  in  the  average  attack  error  can  result 
in  a relatively  large  increase  in  the  percentage  of  lethal  attacks. 

Future  Research  Plans 


A program  for  attaining  maximum  use  of  the  attack  teacher  as  a 
training  device  should  improve  ASW  efficiency  because  the  results  in- 
dicate transfer  of  training  between  the  attack  teacher  and  sea  exercises. 
Since  the  rating  scales  show  a consistent  relationship  to  the  success  of 
ASW  attacks,  the  scales  can  be  used  as  a guide  in  the  coaching  of  oper- 
ators. 
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The  future  research  will  involve  an  experimental  study  of  coach- 
ing as  a means  of  improving  team  performance. 

Additional  research  will  include  an  investigation  of  the  error  re- 
corder being  developed  by  SurASDevDet  to  ascertain  its  value  in  ASW 
training.  The  error  recorder  makes  possible  the  analysis  of  errors 
arising  at  the  QHB  and  the  Range  Recorder. 
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